A Novel Indexing Method for Improving Timeliness of High-Dimensional Data

Investment in information technology (IT) has been growing rapidly and one key reason for investing in IT is to improve information quality (IQ). Timeliness is an important IQ dimension that often needs to be improved for decision making. Especially in the era of big data, timeliness becomes more valued because of challenges of massive data size and high dimensionality. Many financial analyses require timely data to support time-critical decision making. In this paper, we develop a novel index method and effective query algorithms to reduce latency of querying high-dimensional data. The effectiveness of point, range, and similarity queries implemented using our methods is evaluated using a high-dimensional testbed conducted using real world financial data. Results show that our method outperforms existing methods in query speed of three types of queries frequently used in financial decision making.